Understanding the Limits of Capacity Sharing in CMP Private Caches

Chip Multi Processor (CMP) systems present interesting design challenges at the lower levels of the cache hierarchy. Private L2 caches allow easier processor-cache design reuse, thus scaling better than a system with a shared L2 cache, while offering better performance isolation and lower access latency. While some private cache management schemes that utilize space in peer private L2 caches have been recently proposed, we find that there is significant potential for improving their performance. We propose and study an oracular scheme, OPT, which identifies the performance limits of schemes to manage private caches. OPT uses offline-generated traces of cache accesses to uncover applications’ reuse patterns. OPT uses this perfect knowledge of each application’s future memory accesses to optimally place cache blocks brought on-chip in either the local or a remote private L2 cache. We discover that in order to optimally manage private caches, peer private caches must be utilized not only at a local-cache-replacement time, as has been previously proposed, but also at cache-placement time it may be better to place a missed block directly into a peer L2 rather than the traditional approach of first bringing it into the local L2. We implement OPT on a 4-core CMP with 512KB, 8-way, private caches, across 10 carefully chosen, relevant, multiprogram workload mixes. We find that compared to a baseline system that does not employ capacity sharing across private caches, OPT improves weighted-speedup (performance metric) by 13.4% on average. Further, compared to the state of the art technique for private cache management, OPT improves weighted-speedup by 11.2%. This shows the significant potential that exists for improving previously proposed private cache management schemes.